Study on predicting sentiment from images using categorical and sentimental keyword-based image retrieval

Visual stimuli are the most sensitive stimulus to affect human sentiments. Many researches have attempted to find the relationship between visual elements in images and sentimental elements using statistical approaches. In many cases, the range of sentiment that affects humans varies with image categories, such as landscapes, portraits, sports, and still life. Therefore, to enhance the performance of sentiment prediction, an individual prediction model must be established for each image category. However, collecting much ground truth sentiment data is one of the obstacles encountered by studies on this field. In this paper, we propose an approach that acquires a training data set for category classification and predicting sentiments from images. Using this approach, we collect a training data set and establish a predictor for sentiments from images. First, we estimate the image category from a given image, and then we predict the sentiment as coordinates on the arousal–valence space using the predictor of an estimated category. We show that the performance of our approach approximates performance using ground truth data. Based on our experiments, we argue that our approach, which utilizes big data on the web as the training set for predicting content sentiment, is useful for practical purposes

A photomosaic image generation method using photo annotation in a social network environment

With the growing use of social networking services, various applications have been developed to utilize their vast capabilities. Photomosaic techniques, which combine many images to create a new rendering of an input image, can benefit from the capabilities of social networks. In this study, we propose a method that generates a photomosaic image by considering social network context. Our algorithm creates a photomosaic that incorporates photos posted by other users in the users network. We enable the matching function to easily select photos from the albums of users who are connected to the owner of the input image, by computing the closeness of those connections. Moreover, our technique allows the photos in the albums of friends who are annotated in the source image to be matched more effectively

A method for extracting emotion using colors comprise the painting image

Paintings can evoke emotions in viewers. In this paper, we propose a method for extracting emotions from paintings by using the colors that comprise the paintings. The proposed approach is based on a color image scale, which is one of the popular experimental scales focusing on the relation between colors and emotions. We first construct a color combination and emotional word dataset. To this end, we create a color spectrum from the input painting. We then search for the best matching color combination from the dataset, which is most similar to the color spectrum. The best matching color combination is mapped to the corresponding emotional word. Afterward, we extract the emotional word as the emotion evoked by the painting. To evaluate the proposed method, we compared the results of the proposed algorithm to those of a user study on the extraction of emotions from several paintings. Through several experiments, we show that the proposed method exhibits excellent performance with respect to predicting the emotions evoked by a painting. Finally, we propose an image exploration system based on the emotion extraction method mentioned above. In this system, users can explore painting images emotionally coherently

Perceptually Inspired Real-time Artistic Style Transfer for Video Stream

This study presents a real-time texture transfer method for artistic style transfer for video stream. We propose a parallel framework using a T-shaped kernel to enhance the computational performance. With regard to accelerated motion estimation, which is necessarily required for maintaining temporal coherence, we present a method using a downscaled motion field to successfully achieve high real-time performance for texture transfer of video stream. In addition, to enhance the artistic quality, we calculate the level of abstraction using visual saliency and integrate it with the texture transfer algorithm. Thus, our algorithm can stylize video with perceptual enhancements

Human Emotional Care Purposed Automatic Remote Portrait Drawing Generation and Display System Using Wearable Heart Rate Sensor and Smartphone Camera with Depth Perception

We propose a system that automatically generates portrait drawings for the purpose of human emotional care. Our system comprises two parts: a smartphone application and a server. The smartphone application enables the user to take photographs throughout the day while acquiring heart rates from the smartwatch worn by the user. The server collects the photographs and heart rates and displays portrait drawings automatically stylized from the photograph for the most exciting moment of the day. In the system, the user can recall the exciting and happy moment of the day through admiring the drawings and heal the emotion accordingly. To stylize photographs as portrait drawings, we employ nonphotorealistic rendering (NPR) methods, including a portrait etude stylization proposed in this paper. Finally, the effectiveness of our system is demonstrated through user studies

Interactive Illustration of Collage for Children with Folktale E-book

It is always challenging to teach children foreign languages, due to the difficulty of learning and their short attention span. To address the challenge and take advantage of the popularity of touchable tablets and smartphones, we propose an educational folktale e-book (EFE-Book) application with an interactive illustratable tool. EFE-Book is developed to teach preschool children to learn foreign languages by telling folktales with illustrations. To encourage effective learning, EFE-Book provides an interactive collage tool that enables users to create collage-based illustrations by hand. We propose a Voronoi diagram based approach to model paper tiles for the development of EFE-Book. With EFE-Book, the user can create colored paper tiles and attach them to the predesigned sketch through touch interface, such as Apple iPad

Flow-guided hair removal for automated skin lesion identification

In this paper, we propose a method for removing hairs automatically from skin lesion images. To achieve this, we employ an edge-detection technique based on edge-tangent flow. To detect only hair-like structures, rather than contour boundaries, we propose a novel, specialized method for detecting hairs. Regardless of the personal characteristics of the hairs, hairy regions are detected because our method detects coherent thin lines of consistent width. We then restore the hairy regions detected by the proposed method by using the texture synthesis method. Our method restores the regions occluded by hairs with very few remarkable artifacts, because we utilize pixels that actually exist in the source image to restore the occluded areas by searching for the best matching pixels

Photo collage-based photograph display system on mobile computing platform

In the last few decades, mobile computing platform technology has grown rapidly, as observed from smart phones that have quickly become ubiquitous. The mobile computing platform is the most widely used platform in our life today, and digital photographs captured through these devices have become routine for most people. In this study, we propose a novel artistic method for displaying photographs in mobile devices as a photo collage. Using our system, users can view a representative photograph as a collage of photographs associated with a certain event and access each of photographs individually. To implement this, we employ centroidal Voronoi diagram to obtain an even distribution of tiles, and use the sites as the location of tiles. We use the edge avoidance technique to prevent tiles from being located across the edges. To obtain the direction of tiles that follow near a strong edge, we employ the Edge tangent Flow field and use the field as the directions of tiles. Finally, we search for photographs that best match the tiles calculated above by using a thumbnail difference metric

Interactive Painterly Rendering for Mobile Devices

Part 3: PostersInternational audiencePainterly rendering is among the most popular non-photorealistic rendering techniques and has been employed in many applications. Research on painterly rendering mainly focuses on automatically generating artistic results. In this study, we aim to develop an entertaining and interactive application for painterly rendering with touchscreen mobile devices. The proposed application provides user interaction for added enjoyment and ensures high-quality painterly results. We provide a method for finding the appropriate position of the brush stroke around the points touched by the user and for generating and rendering the brush stroke. With the proposed method, users can quickly generate high-quality painterly results